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Abstract 

We  developed  an  integrated  methodology  for  uncertainty  quantification  (UQ)  that  proceeds 
from  initial  problem  definition  to  engineering  applications.  Towards  this  goal,  we  worked  on 
five  research  areas:  (1)  Mathematical  analysis  of  SPDEs  and  multiscale  formulation;  (2)  Nu¬ 
merical  solution  of  SPDEs;  (3)  Reduced-Order  modeling;  (4)  Estimation/Inverse  problems;  and 
(5)  Robust  optimization  and  control.  This  work  set  the  mathematical  foundations  of  Uncer¬ 
tainty  Qantification  methods  used  by  many  diverse  communities  in  computational  mechanics, 
fluid  dynamics,  plasma  dynamics,  and  materials  science.  We  have  pioneered  methods  for  efficient 
high-dimensional  representations  of  stochastic  processes,  established  Wick-Malliavin  approxima¬ 
tion  for  nonlinear  SPDEs,  theoretical  error  estimates  for  multiscale  parametric  and  stochastic 
PDEs,  a  new  approach  to  design  of  experiment  and  UQ  on  parametric  manifolds,  multi-fidelity 
optimization-under-uncertainty,  a  data-driven  Bayasian  framework  and  probabilistic  graphical 
models  for  UQ,  and  information-based  coarse  graining  methods.  We  have  also  demonstrated 
an  integration  of  our  UQ  methodology  and  all  five  areas  for  a  benchmark  problem.  We  have 
published  more  than  150  papers  in  top  mathematical  journals,  obtained  one  patent  (MIT), 
and  have  established  one  software  company  (MIT). 
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1  Status/Progress 

In  the  following  we  provide  some  research  highlights  in  each  of  the  five  research  areas  of  the  MURI. 

1.1  Mathematical  analysis  of  SPDEs  and  multiscale  formulation 

Wick  Malliavin  Approximations  to  nonlinear  Stochastic  PDEs  (Brown  leads)  An  im¬ 
portant  achievement  is  the  development  of  completely  new  nonlinear  Malliavin  calculus.  This  type 
of  calculus  is  important  for  the  analysis  and  simulation  of  stationary  and/or  “causal”  systems.  It 
allows  effective  treatment  of  systems  perturbed  by  nonlinear  functions  of  colored  Gaussian  noise. 
We  have  also  developed  an  effective  methodology  for  homogenization  of  random  elliptic  PDEs  with 
deterministic  multi-scale  structure  of  the  coefficients. 

Approximating  nonlinearities  in  Stochastic  Partial  Differential  Equations  (SPDEs)  via  the  Wick 
product  has  often  been  used  in  quantum  field  theory  and  stochastic  analysis.  The  main  benefit  is 
simplification  of  the  equations  but  at  the  expense  of  introducing  modeling  errors.  We  have  shown 
that  the  Wick  solutions  have  accuracy  comparable  to  linear  stochastic  perturbation  series  solutions. 
However,  number-theoretical  renormalizations,  e.g.,  based  on  Catalan  numbers  for  the  Wick-Navier- 
Stokes  (WNS)  equations,  can  improve  the  accuracy  by  orders  of  magnitude.  The  propagator  of 
Wick  approximations  to  nonlinear  SPDEs  has  the  same  structure  as  the  system  of  equations  for  the 
coefficients  of  formal  power  series  solutions.  Moreover,  the  structure  of  this  propagator  is  seemingly 
universal,  i.e. ,  independent  of  the  type  of  noise.  We  also  introduced  new  high-order  stochastic  ap¬ 
proximations  via  Wick-Malliavin  series  expansions  for  Gaussian  and  uniformly  distributed  noises, 
and  demonstrate  convergence  as  the  number  of  expansion  terms  increases.  Our  results  are  for  Burg¬ 
ers  and  Navier-Stokes  (NS)  equations  but  the  same  approach  can  be  adopted  for  other  nonlinear 
SPDEs  with  polynomial  nonlinearities  and  more  general  noises. 

Multiscale  Data-Driven  Methods  (Caltech  leads)  We  made  significant  progress  in  developing 
effective  numerical  methods  for  solving  stochastic  partial  differential  equations  and  multiscale  prob¬ 
lems.  In  particular,  we  have  developed  (i)  a  dynamically  bi-orthogonal  method  for  time-dependent 
SPDEs;  (ii)  a  data-driven  stochastic  method  for  multi-query  stochastic  problems;  (iii)  a  multiscale 
model  reduction  method  for  PDEs  with  nonseparable  multiscale  solutions.  We  have  also  made 
progress  in  deriving  a  multiscale  closure  for  the  3D  incompressible  Navier-Stokes  equations  and 
in  developing  data-driven  time-frequency  analysis  by  exploiting  the  intrinsic  sparse  structure  of 
multiscale  data.  Below  we  will  give  some  details  on  project  (i): 

We  proposed  a  dynamically  bi-orthogonal  method  (BO)  to  study  time  dependent  SPDEs.  This 
was  inspired  by  ongoing  work  in  the  MURI  on  dyamicallly  orthogonal  expansions  (DO).  The  ob¬ 
jective  of  both  BO  and  DO  methods  is  to  exploit  some  intrinsic  sparse  structure  in  the  stochastic 
solution  by  constructing  the  sparsest  representa-  tion  of  the  stochastic  solution  via  a  bi-orthogonal 
basis.  These  methods  essentially  track  the  Karhunen-Loeve  expansion  dynamically  without  the 
need  to  form  the  covariance  matrix  or  to  compute  its  eigen-decomposition.  In  the  first  part  of  the 
work,  we  derived  the  dynamically  bi-orthogonal  formulation  for  SPDEs,  discussed  several  theoret¬ 
ical  issues,  such  as  the  dynamic  bi-orthogonality  preservation  and  the  consistency  between  the  BO 
formulation  and  the  original  SPDE.We  also  gave  some  numerical  implementation  details  of  the  BO 
methods,  including  the  representation  of  stochastic  basis  and  techniques  to  deal  with  eigenvalue 
crossing.  In  the  second  part,  we  presented  an  adaptive  strategy  to  dynamically  remove  or  add 
modes,  performed  a  detailed  complexity  analysis,  proposed  a  parallel  implementation  of  DyBO, 
and  discussed  various  generalizations  of  this  approach.  We  have  applied  the  BO  method  to  solve 
the  ID  stochastic  Burgers  equation,  2D  incompressible  Navier-Stokes  equations  and  the  Boussinesq 
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approximation  with  Brownian  motion  forcings.  These  numerical  examples  demonstrate  that  the 
BO  method  solves  these  nonlinear  time-dependent  SPDEs  accurately  and  efficiently.  In  subsequent 
work,  the  group  at  Brown  derived  an  exact  equivalence  between  the  BO  and  DO  methods  and 
developed  a  hybrid  approach  that  combines  the  best  computational  features  of  both  methods. 

1.2  Numerical  solution  of  SPDEs 

We  devloped  methods  to  solve  the  Navier-Stokes  and  other  nonlinear  SPDEs  in  more  than  100 
dimensions!  Several  other  dvelopments  include  advances  in  the  generalized  polynomial  chaos  and 
its  variance  as  well  as  in  Bayesian  type  methods.  In  the  following,  we  provide  a  partial  list  and  in 
the  references  we  provide  all  methods  we  have  devloped  in  this  MURI. 

The  Cornell  Pis  focused  on  sparse  Bayesian  kernel  techniques  (relevance  vector  machines,  RVM) 
for  the  solution  of  SPDEs.  Each  dimension  of  the  multivariate  response  was  modeled  using  local 
kernels  centered  on  top  of  each  data  point.  The  missing  scale  parameters  of  the  kernels  were  selected 
by  maximizing  the  joint  marginal  likelihood  of  all  dimensions  of  the  response.  To  address  issues 
with  high  dimensionality  and  non-informat ive  variance,  these  algorithms  were  later  extended  to 
weighted  mixture  of  Gaussian  processes. 

The  Brown  group  focused  on  high-dinrensional  problems  using  adaptive  AN OVA  with  applica¬ 
tions  to  the  performance  analysis  of  the  horn  problem,  fluid  flows  and  electromagnetic  scattering, 
and  also  in  developing  new  polynomial  chaos  methods  for  white  noise.  In  the  context  of  the  horn 
benchmark,  AN OVA  was  combined  with  the  reduced  basis  method  to  enable  a  similar  parametric 
reduction  of  the  high  dimensional  problem  to  allow  the  development  of  a  certified  reduced  ba¬ 
sis  methods  for  the  critical  components  of  systems  with  many  scattering  bodies.  This  allowed 
for  the  development  of  reduced  basis  methods  for  problems  with  many  parameters  as  has  been 
demonstrated  for  the  acoustic  horn  problem  (see  below).  Another  approach  to  tackling  the  curse- 
of-dimensionality  is  the  formulation  of  PDF  equations  for  colored  noise  (joint  solution-excitation; 
fractional  PDEs)  and  solve  them  using  ANOVA  or  Proper  Generalized  Decomposition. 

The  Caltech  group  worked  on  two  methods:  Data-Driven  Stochastic  Multiscale  Method  (DSM) 
and  Multiscale  Multi-Level  Monte  Carlo  Method  (MsMLMC),  respectively.  The  second  method 
can  be  incorporated  into  the  first  to  boost  its  applicability  and  efficiency,  especially  for  tough 
problems  involving  randomness  and  multiscales  simultaneously.  An  important  aspect  of  DSM  is 
the  re-usability  of  the  constructed  stochastic  basis  for  different  deterministic  forcing  functions. 
For  computational  efficiency,  a  low-rank  approximation  method  is  used  (developed  in  compressed 
sensing)  to  exploit  the  low-rank  structure  of  the  covariance  matrix.  Both  DSM  and  MsMLMC  have 
been  applied  to  the  horn  benchmark  with  the  latter  giving  up  to  100  times  speed-up  compared  to 
standard  MC. 

Uncertainty  Quantification  for  Multiscale  PDEs  using  a  Graph  Theoretic  Approach 
(Cornell  leads)  We  developed  a  probabilistic  graphical  model  based  methodology  to  efficiently 
perform  uncertainty  quantification  in  the  presence  of  both  stochastic  input  and  multiple  scales. 
Both  the  stochastic  input  and  model  responses  were  treated  as  random  variables  in  this  frame¬ 
work.  Their  relationships  were  modeled  by  graphical  models  which  give  explicit  factorization  of 
a  high-dinrensional  joint  probability  distribution.  The  hyperparameters  in  the  probabilistic  model 
were  learned  using  sequential  Monte  Carlo  (SMC)  method  locally  in  the  graph.  Coarse  graining 
(stochastic  homogenization)  was  addressed  in  a  non  parametric  way  using  hidden  variables  in  a  way 
that  naturally  arises  within  the  Bayesian  graph  theoretic  framework.  Finally,  we  made  predictions 
from  the  probabilistic  graphical  model  using  the  belief  propagation  algorithm  rather  than  Monte 
Carlo  integration.  Belief  propagation  has  the  potential  of  almost  linear  scaling  in  certain  applica- 
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tions.  Numerical  examples  were  investigated  to  show  the  accuracy  and  efficiency  of  the  predictive 
capability  of  the  developed  graphical  model.  Many  interesting  extensions  of  this  framework  were 
investigated  that  potentially  could  lead  to  a  transformative  way  for  UQ  in  multiscale/multiphysics 
PDE  systems. 

Information  Theoretic  Coarse  Graining:  Relative  entropy  (Cornell  leads)  Relative  en¬ 
tropy  has  been  shown  to  provide  a  principled  framework  for  the  selection  of  coarse-grained  po¬ 
tentials.  Despite  the  intellectual  appeal  of  it,  its  application  has  been  limited  by  the  fact  that  it 
requires  the  solution  of  an  optimization  problem  with  noisy  gradients.  When  using  deterministic 
optimization  schemes,  one  is  forced  to  either  decrease  the  noise  by  adequate  sampling  or  to  resolve 
to  ad  hoc  modifications  in  order  to  avoid  instabilities.  The  former  increases  the  computational 
demand  of  the  method  while  the  latter  is  of  questionable  validity.  In  order  to  address  these  issues 
and  make  relative  entropy  widely  applicable,  we  proposed  alternative  schemes  for  the  solution  of 
the  optimization  problem  using  stochastic  algorithms.  Cluster  expansions  are  simplified,  Ising-like 
models  for  binary  alloys  in  which  vibrational  and  electronic  degrees  of  freedom  are  coarse  grained. 
The  usual  practice  is  to  learn  the  parameters  of  the  cluster  expansion  by  fitting  the  energy  they 
predict  to  a  finite  set  of  ab  initio  calculations.  In  some  cases,  experiments  suggest  that  such  ap¬ 
proaches  may  lead  to  overestimation  of  the  phase  transition  temperature.  We  presented  a  novel 
approach  to  fitting  the  parameters  based  on  the  relative  entropy  framework  which,  instead  of  en¬ 
ergies,  attempts  to  fit  the  Boltzmann  distribution  of  the  configurational  degrees  of  freedom.  We 
showed  how  this  leads  to  T-dependent  parameters. 

Numerical  Methods  for  High-Dimensional  PDF  Equations  (Brown  leads)  In  this  task 
we  addressed  the  problem  of  computing  the  numerical  solution  to  kinetic  partial  differential  equa¬ 
tions  involving  many  phase  variables.  These  types  of  equations  arise  naturally  in  many  differ¬ 
ent  areas  of  mathematical  physics,  e.g.,  in  particle  systems  (Liouville  and  Boltzmann  equations), 
stochastic  dynamical  systems  (FokkerPlanck  and  DostupovPugachev  equations),  random  wave  the¬ 
ory  (MalakhovSaichev  equations)  and  coarse-grained  stochastic  systems  (MoriZwanzig  equations). 
We  proposed  three  different  classes  of  new  algorithms  addressing  high-dimensionality:  The  first  one 
is  based  on  separated  series  expansions  resulting  in  a  sequence  of  low-dimensional  problems  that 
can  be  solved  recursively  and  in  parallel  by  using  alternating  direction  methods.  The  second  class 
of  algorithms  relies  on  truncation  of  interaction  in  low-orders  that  resembles  the  BogoliubovBorn- 
GreenKirkwoodYvon  (BBGKY)  framework  of  kinetic  gas  theory  and  it  yields  a  hierarchy  of  coupled 
probability  density  function  equations.  The  third  class  of  algorithms  is  based  on  high-dimensional 
model  representations,  e.g.,  the  ANOVA  method  and  probabilistic  collocation  methods.  A  common 
feature  of  all  these  approaches  is  that  they  are  reducible  to  the  problem  of  computing  the  solution 
to  high-dinrensional  equations  via  a  sequence  of  low-dimensional  problems.  The  effectiveness  of  the 
new  algorithms  was  demonstrated  in  numerical  examples  involving  nonlinear  stochastic  dynamical 
systems  and  partial  differential  equations,  with  up  to  120  variables. 

An  adaptive  hybrid  bi-orthogonal/dynamically-orthogonal  method  for  the  stochastic 
Navier-Stokes  equations  (Brown  leads)  A  new  hybrid  methodology  for  SPDEs  was  developed 
based  on  the  dynamically  orthogonal  (DO)  and  bi-orthogonal  (BO)  methods;  both  approaches  are 
an  extension  of  the  Karhunen-Lo‘eve  (KL)  expansion  and  hence  they  capture  a  low-dimensional 
structure  of  the  solution  by  tracking  the  KL  expansion  of  the  solution  at  any  given  time  on-the- 
fly.  It  has  been  shown  that  DO  and  BO  are  equivalent  in  the  sense  that  one  method  is  an  exact 
reformulation  of  the  other  through  a  matrix  differential  equation.  However,  DO  suffers  numerically 
when  there  is  a  high  condition  number  of  the  covariance  matrix  while  BO  suffers  when  there  is 
an  eigenvalue  crossing.  To  this  end,  we  proposed  a  unified  hybrid  framework  of  the  two  methods 
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by  utilizing  an  invertible  and  linear  transformation  between  them.  We  also  presented  an  adaptive 
algorithm  to  add  or  remove  modes  dynamically  to  better  capture  the  transient  behavior.  Several 
numerical  examples  including  the  Navier-Stokes  equations  were  presented  to  illustrate  this  new 
adaptive  hybrid  BO-DO  method. 

1.3  Reduced-Order  modeling 
Certified  Basis  (MIT  leads) 

Uncertainty  Quantification  (UQ)  in  almost  all  forms  and  approaches  is  perforce  a  many-query 
context.  It  thus  follows  that  UQ  is  very  well  suited  to  the  offline-online  strategy  afforded  by  model 
order  reduction  (MOR)  techniques.  In  the  past,  application  of  MOR  to  UQ  has  been  inhibited 
by  fundamental  restrictions  in  MOR  methodology:  MOR  could  treat  only  a  rather  limited  class  of 
partial  differential  equations  (PDEs);  MOR  could  treat  problems  characterized  by  only  relatively 
few  parameters.  In  our  MURI  effort  we  have  substantially  expanded  the  capabilities  of  MOR 
techniques  and  furthermore  proposed  new  frameworks  in  which  MOR  can  serve  well  the  goals  of 
UQ.  We  have  also  further  improved  our  a  posteriori  error  estimators  so  as  to  better  assess  and 
control  the  “self-uncertainty”  introduced  by  model  order  truncation. 

Our  accomplishments  in  the  area  of  expanded  classes  of  problems  focus  on  coupled  and  nonlinear 
problems,  as  well  as  implementations  for  supercomputers  but  also  deployed  platforms.  Our  accom¬ 
plishments  in  the  area  of  increased  parameter  dimensionality  are  twofold:  related  to  parameter- 
domain  decomposition,  the  development  of  an  uh-pv  reduced  basis  approximation;  related  to  spatial- 
domain  decomposition,  the  development  of  the  port-reduced  static-condensation  reduced-basis  el¬ 
ement  (PR-SCRBE)  method  for  linear  problems  and  also  eigenproblems.  Our  accomplishments 
in  the  area  of  frameworks  for  uncertainty  quantification  focus  on  data  assimilation  for  state  esti¬ 
mation.  Finally,  our  accomplishments  in  the  area  of  a  posteriori  error  estimation  focus  on  exact 
bounds,  and  on  space-time  techniques  for  long-time  evolution  problems. 

We  consider  these  accomplishments  in  more  detail  below  and  more  details  can  be  found  in  the 
publications  of  Patera’s  group. 

Al.  Expanded  problem  classes.  We  have  expanded  the  reduced  basis  method  to  consider  long¬ 
time  evolution  of  nonlinear  problems  (such  as  the  incompressible  Navier-Stokes  equations),  and 
also  coupled  problems  (convection-conduction)  related  to  heat  exchanger  design.  We  have  also 
expanded  the  reach  of  our  approach  by  considering  optimized  implementations  of  the  reduced  basis 
method  on  supercomputers  and  also  deployed  platforms  (for  real-time  computation). 

A2.  Treatment  of  many  parameters.  Most  of  this  effort  falls  within  two  methodological  thrusts. 

The  “h-p”  reduced  basis  method.  This  breaks  the  parameter  domain  into  optimal  parame¬ 
ter  subdomains  and  then  applies  the  reduced-basis  method  on  each  parameter  subdomain.  The 
method  permits  higher  parameter  dimensionality  due  to  the  smaller  domain  (hence  less  rich  solution 
variation)  associated  with  each  approximation. 

The  Port-Reduced  Static  Condensation  Reduced-Basis  Element  Method  (PR-SCRBE).  The 
PR-SCRBE  approach  is  a  component-based  system  synthesis  approach  which  exploits  model  order 
reduction  at  two  levels:  at  the  level  of  ports  (the  interfaces  at  which  components  connect),  informed 
by  evanescence  arguments;  at  the  level  of  the  component  interiors,  informed  by  low-dinrensional 
parametric  manifolds.  The  method  can  address  many  parameters,  and  indeed  also  topology  vari¬ 
ations,  thanks  to  the  component  decomposition  and  associated  divide-and-conquer  strategy:  we 
solve  many  problems  with  a  few  parameters  rather  than  one  problem  with  many  parameters.  The 
method  is  also  equipped  with  error  estimators  both  for  the  port  and  interior  truncations. 

A3.  Data  Assimilation.  Most  of  this  effort  has  been  focused  on  the  development  of  a  new 
data  assimilation  approach,  the  Parametrized  Background  Data  Weak  method,  and  an  associated 
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rigorous  theory  of  stability  and  approximation.  The  distinguishing  feature  of  the  method  is  the 
effective  incorporation  of  low-dimensional  parameter  manifolds  identified,  and  approximated,  by 
methods  developed  within  the  context  of  reduced  basis  methods.  The  PBDW  method  is  non- 
intrusive  in  the  sense  that  the  PDE  appears  only  in  the  offline  stage  and  furthermore  provides 
real-time  response  in  the  online  stage.  The  method  has  been  applied  within  our  group  to  physical 
systems,  in  particular  to  acoustics  experiments,  with  considerable  predictive  success. 

Alh  Improved  a  posteriori  error  estimators.  Most  of  this  effort  falls  within  two  methodological 
thrusts. 

The  first  thrust  is  the  development  of  a  formulation  in  which  the  reduced-basis  error  bound  is 
measured  with  respect  to  the  exact  solution  of  the  PDE.  In  earlier  approaches,  the  reduced-basis 
error  bound  is  measured  relative  to  a  highly  refined  “truth”  finite  element  approximation.  The  new 
approach,  proposed  and  developed  by  Dr  Masa  Yano,  is  preferred  not  only  for  the  increased  rigor, 
but  also  because  the  formulation  naturally  suggests  a  simultaneous  finite-element  reduced-basis 
adaptive  refinement  strategy. 

The  second  thrust  is  the  development  of  improved  error  estimates  for  weakly  stable  evolution 
problems.  In  the  past,  error  bounds  for  weakly  stable  evolution  problems  exhibited  exponential 
growth  such  that  only  short-time  estimates  were  meaningful.  In  the  new  approach,  we  consider 
a  space-time  formulation  informed  by  an  optimal  inf-sup  parameter  which  considers  worst-case 
growth  not  from  timestep  to  timestep  but  rather  over  the  entire  time  interval  and  consistent  with 
the  governing  equation:  long-time  evolution  may  thus  be  pursued. 

Stochastic/Multiscale  UQ  for  Wave  Dynamics  (Brown  leads)  In  this  research  task  we 
focused  on  three  separate  but  connected  efforts.  The  continued  development  of  certified  reduced 
basis  methods  in  general  and  with  a  particular  focus  on  wave  problems.  We  have  demonstrated  the 
effectiveness  of  such  methods  for  a  variety  of  problem  types,  including  parameterized  geometries 
and  the  use  of  such  models  for  uncertainty  quantification  during  scattering.  A  substantial  effort 
has  been  in  the  development  and  application  of  certified  reduced  methods  for  integral  equations, 
including  the  development  of  methods  that  allows  the  computation  of  scattering  by  a  collection 
of  scatterers.  The  challenges  associated  with  the  development  of  reduced  models  for  parametrized 
models  with  a  high-dimensional  parameter  space  has  also  been  considered.  We  have  developed 
methods  that  dramatically  accelerate  the  greedy  approximation  in  the  reduced  basis  development 
and  demonstrated  the  ability  to  handle  problems  with  many  parameters.  In  a  related  work  we  have 
shown  how  to  combine  reduced  models  with  AN OVA  expansions  to  allow  the  effective  estimation 
of  parametric  sensitivity,  leading  to  parameter  compression  to  allow  the  development  of  a  reduced 
model  for  relevant  parameters  only.  A  major  part  of  this  effort  has  been  devoted  to  the  development 
and  analysis  of  high-order  accurate  multi-scale  finite  element  methods.  The  work  is  based  on  a 
new  and  more  direct  high-order  multi-scale  expression  and  the  analysis  confirms  optimal  behavior. 
We  have  considered  both  the  classic  Poisson  problem  as  well  as  completed  the  first  analysis  of 
multi-scale  finite  element  methods  for  the  wave  Helmholtz  equation.  We  have  also  demonstrated 
how  reduced  basis  methods  can  be  used  to  reduce  the  computational  overhead  associated  with 
heterogeneous  multi-scale  behavior. 

1.4  Estimation/Inverse  problems 
Bayesian  Techniques  (Cornell  leads) 

Fully  Bayesian  Uncertainty  Quantification  Framework/Gaussian  Processes  with  corre¬ 
lated  outputs:  Computer  codes  simulating  physical  systems  usually  have  responses  that  consist 
of  a  set  of  distinct  outputs  (e.g.,  velocities  and  pressures)  that  evolve  also  in  space  and  time  and 
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depend  on  many  unknown  input  parameters  (e.g.,  physical  constants,  initial/boundary  conditions 
etc.).  Furthermore,  essential  engineering  procedures  such  as  UQ,  inverse  problems  or  design  are 
notoriously  difficult  to  carry  out  mostly  due  to  the  limited  simulations  available.  The  aim  of  this 
work  was  to  introduce  a  fully  Bayesian  approach  for  treating  these  problems  which  accounts  for 
the  uncertainty  induced  by  the  finite  number  of  observations.  Our  model  was  built  on  a  multi¬ 
dimensional  Gaussian  process  that  explicitly  treats  correlations  between  distinct  output  variables 
as  well  as  space  and/or  time.  The  proper  use  of  a  separable  covariance  function  enabled  us  to 
describe  the  huge  covariance  matrix  as  Kronecker  product  of  smaller  ones  leading  to  efficient  algo¬ 
rithms  for  carrying  out  inference  and  predictions.  The  novelty  of  this  work  is  the  recognition  that 
the  Gaussian  process  model  actually  defines  a  posterior  probability  measure  on  the  function  space 
of  possible  surrogates  for  the  computer  code  and  the  derivation  of  an  algorithmic  procedure  that 
allows  us  to  sample  it  efficiently.  We  demonstrated  how  the  scheme  can  be  used  in  uncertainty 
quantification  tasks  in  order  to  obtain  error  bars  for  the  statistics  of  interest  that  account  for  the 
finite  number  of  observations. 

Sparse  Bayesian  Techniques.  Multi-output  sparse  Bayesian  techniques  (extension  of  relevance 
vector  machines)  that  are  able  to  automatically  identify  the  most  relevant  of  a  set  of  basis  functions 
(using  either  localized  kernel  functions  or  an  optimal  orthogonal  polynomial  basis).  When  using 
an  optimal  orthogonal  polynomials  basis,  our  techniques  may  be  thought  as  a  Bayesian,  tree-based 
extension  of  generalized  Polynomial  Chaos  (gPC).  This  combines  the  optimal  convergence  of  the 
gPC  for  smooth  functions,  with  locality  capturing  discontinuities  and  the  Bayesian  framework 
allowing  the  quantificaiton  of  epistemic  uncertainty.  Our  numerical  experiments  demonstrated 
that  this  is  a  powerfull  combination.  The  sparsity  of  the  resulting  surrogate:  1)  may  be  intuitively 
interpretable  and  2)  are  super-fast  to  evaluate. 

Treed  Multi-output  Gaussian  Process:  We  developed  an  efficient,  Bayesian  Uncertainty  Quan¬ 
tification  framework  using  a  novel  treed  Gaussian  process  model.  The  tree  is  adaptively  constructed 
using  information  conveyed  by  the  observed  data  about  the  length  scales  of  the  underlying  process. 
On  each  leaf  of  the  tree,  we  utilize  Bayesian  Experimental  Design  techniques  in  order  to  learn  a 
multi-output  Gaussian  process.  The  constructed  surrogate  can  provide  analytical  point  estimates, 
as  well  as  error  bars,  for  the  statistics  of  interest.  We  numerically  demonstrated  the  effectiveness  of 
the  suggested  framework  in  identifying  discontinuities,  local  features  and  unimportant  dimensions 
in  the  solution  of  SPDEs. 

Solution  of  Inverse  Problems  with  Limited  Forward  Solver  Evaluations:  A  Bayesian 
Perspective.  Solving  inverse  problems  based  on  computationally  demanding  forward  solvers  is 
ubiquitously  difficult  since  one  is  necessarily  limited  to  just  a  few  observations  of  the  response  sur¬ 
face.  This  limited  information  induces  additional  uncertainties  on  the  posterior  distributions.  The 
main  contribution  of  this  work  is  the  reformulation  of  the  solution  of  the  inverse  problem  when  the 
expensive  forward  model  is  replaced  by  a  set  of  simulations.  The  proposed  solution  is  based  on 
the  idea  of  a  Bayesian  surrogate  that  replaces  the  code.  We  derived  three  approximations  of  the 
reformulated  solution  with  increasing  complexity  and  fidelity.  We  demonstrated  numerically,  that 
the  proposed  approximations  indeed  capture  the  epistemic  uncertainty  on  the  solution  of  inverse 
problem  induced  by  the  fact  that  the  forward  model  is  replaced  by  a  set  of  simulations  and  that 
they  converge  to  the  true  solution  as  the  number  of  simulations  is  increased. 


1.5  Robust  optimization  and  control 
Design  under  Uncertainty  (MIT  leads) 
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Under  this  thrust  we  have  pursued  two  main  research  topics:  multifidelity  methods  to  acceler¬ 
ate  the  cost  of  solving  optimization  under  uncertainty  problems,  and  a  goal-oriented  approach  to 
inference  of  distributed  parameters.  Our  multifidelity  approaches  build  on  the  methods,  tools  and 
applications  developed  in  the  Model  Reduction  thrust  of  the  MURI  project. 

In  optimization  under  uncertainty  problems,  computing  the  mean,  variance,  or  other  statistics  of 
the  high-fidelity  model  output  for  every  change  in  the  design  variables  is  computationally  expensive 
due  to  the  large  number  of  model  evaluations  needed.  In  many  practical  situations,  a  low-fidelity 
model  is  available  to  provide  useful  information  about  the  output  of  the  high-fidelity  model  at  a 
lower  cost.  Multifidelity  Monte  Carlo  simulation  is  a  modification  of  the  control  variate  method 
that  takes  advantage  of  the  correlation  between  the  output  of  the  low-fidelity  model  and  the  output 
of  the  high-fidelity  model  to  reduce  the  computational  cost  of  uncertainty  propagation. 

Engineered  systems  parametrized  by  distributed  quantities  represent  a  significant  challenge  for 
state-of-the-art  computational  methods  and  inverse  problem  formulations.  An  infinite-dimensional 
parameter  is  identified  to  predict  output  quantities  of  interest.  Goal-oriented  inference  is  the  process 
by  which  these  final  outputs  are  exploited  in  the  inference  process.  In  the  linear  case,  we  have  shown 
that  the  inference  algorithm  can  be  suitably  modified  to  improve  online  efficiency  without  sacrificing 
accuracy  in  prediction  of  outputs.  Our  work  focused  on  extending  goal-oriented  inference  to  the 
setting  of  nonlinear  problems.  Our  work  on  the  deterministic  inverse  problem  formulation  has 
focused  on  employing  error  estimation  techniques  popular  in  the  mesh  adaptation  community  to 
obtain  a  parameter  estimate  that  has  the  correct  prediction,  but  without  converging  the  parameter. 
In  the  statistical  setting,  we  extended  recent  work  in  Bayesian  inference  to  identify  a  map  from 
prior  predictive  to  posterior  predictive.  One  then  would  obtain  samples  of  the  posterior  predictive 
directly  from  applications  of  the  prediction  model  to  prior  samples  and  propagation  through  the 
identified  map.  The  required  map  will  generally  have  many  fewer  parameters  than  the  analogous 
map  from  prior  to  posterior  since  it  is  applied  in  the  prediction  space. 

1.6  Integrated  UQ  Methodology 

In  order  to  demonstrate  advances  on  all  five  research  areas  we  focused  on  the  specific  horn  bench¬ 
mark  problem,  hence  addressing  the  design  of  wave-dominated  problems  under  uncertainty.  We 
tested  new  developments  on  nonlinear  Malliavin  calculus,  combining  reduced  basis  methods  with 
ANOVA,  model  validation,  on  quantifying  model  uncertainty  in  inverse  problems,  on  stochastic 
quantization  for  the  Navier-Stokes  equations,  on  learning  techniques,  and  on  stochastic  multiscale 
modeling  of  materials. 

We  considered  a  frequency-domain  acoustic  planar  horn  problem  first  introduced  and  analyzed 
in  (Udawalpola  &  Berggren,  I.  J.  Num.  Meth.  Eng.,  73:1571,  2008)  in  the  deterministic  framework. 
The  horn  consists  of  a  straight  waveguide  followed  by  a  flare  section.  The  pressure  field  satisfies  a 
Helmholtz  equation  with  an  incoming- wave  condition  at  the  waveguide  inlet,  zero-flow  (Neumann) 
conditions  on  the  horn  walls,  and  a  radiation  condition  for  the  farfield.  We  took  for  our  input 
(stochastic)  parameters  the  flare  geometry,  wavenumber  k,  and  impedances  Zt,  Z b  of  the  top  and 
bottom  flare  walls;  we  also  must  specify  a  parameter  domain.  We  took  for  our  output  the  effective 
reflection  coefficient  of  the  horn;  the  aim  is  to  minimize  reflection  and  hence  maximize  power 
transmission.  First,  we  obtained  a  “truth”  finite  element  (FE)  approximation  and  subsequently 
we  developed  a  reduced  basis  (RB)  approximation  to  the  FE  approximation  with  corresponding 
rigorous  a  posteriori  error  bounds  for  the  difference  between  the  finite  element  and  reduced  basis 
output  predictions.  The  results  were  reported  in  a  previous  progress  report  (2011). 
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2  Personnel  Supported  During  Duration  of  Grant 

•  Faculty:  Karniadakis,  Hesthaven,  Rozovsky,  Zabaras,  Hou,  Patera,  Willcox. 

•  Postdocs  (partial  support):  (Brown)  D.  Venturi,  L.  Grinberg,  S.  Zhang;  B.  Stamm,  P.  Gatto,  S. 
Zhang, F.  Zeng;  (Caltech)  Z.  Zhang,  F-N.  Hwang,  C.  Liu,  Z.  Shi,  Q.  Li,  G.  Luo;  (MIT)  J.  Eftang, 
D.B.P.  Huynh,  J.D.  Penn,  D.J.  Knezevic,  K.  Smetana,  T.  Tonn,  S.  Vallaghe,  M.  Yano,  T.  Tom,  M. 
Yano,  Q.  Liao  . 

•  PhD  Students  (partial  support):  14  at  Brown;  6  at  Cornell;  6  at  Caltech;  3  at  MIT. 


3  Honors  &  Awards 

•  Hesthaven  -  SIAM  Fellow  (2014). 

•  Hou  -  AMS  Fellow  (2012);  SIAM  Fellow  (2009);  AAAS  (2011). 

•  Karniadakis,  SIAM  Fellow  (2010);  USACM  T.J.  Oden  Medal  (2013);  R.  Kleinmann  SIAM  award 
(2015). 

•  Patera  -  USACM  T.J.R.  Hughes  Medal  (2013);  Hans  Kupczyk  Guest  Professorship  Award  2010 
from  the  University  of  Ulm  (Germany);  Honorary  Member,  Societe  de  Mathematiques  Appliquees 
et  Industrielles  (SMAI,  France),  2012;  Chaire  d’Excellence  (Senior  Research  Chair),  Fondation 
Sciences  Mathematiques  de  Paris,  France,  2013-2015. 

•  Wilcox  and  Leo  Ng  (PhD  student  with  Willcox)  received  second  place  in  the  AIAA  Multidisci¬ 
plinary  Analysis  and  Optimization  conference  Student  Paper  competition,  for  his  paper  entitled 
” Multifidelity  Uncertainty  Propagation  for  Optimization  Under  Uncertainty.”  (September  2012). 

4  AFRL  Point  of  Contact 

•  Jordan,  Jennifer  L  Dr  CIV  USAF  AFMC  AFRL/RWM  ,  Energetic  Materials  Core  Technical 
Competency  Lead,  Eglin  Munitions  Directorate  visited  Cornell  and  research  collaborative  plans 
are  under  way. 

•  Willcox  discussed  multifidelity  methods  with  Ray  Kolonay  and  Ed  Alyanak  from  AFRL. 

•  Philip  Beran,  WPAFB,  OH  45433,  Phone  937-255-665.  Visited  and  gave  a  talk  at  MIT  in  Fall 
2010; 

•  Horie  Yasuyuki,  CIV  USAF  AFMC  AFRL/RWMER  (Yasuyuki.Horie@eglin.af.mil);  Dutton,  Rol- 
lie  E  Civ  USAF  AFMC  AFRL/RXLMP  (Rollie.Dutton@wpafb.af.mil);  Cooper,  William  L  Dr  CIV 
USAF  AFMC  AFRL/RWMWH  (william.cooper@eglin.af.mil).  A  short  course  on  UQ  was  given  by 
co-PI  Zabaras  at  Eglin  AFB  and  also  at  General  Electric. 

5  Transitions 

•  Karniadakis’s  group  -  http://sourceforge.net/projects/mepcmpackage/  The  Multi-Element  Prob¬ 
abilistic  Collocation  Method  Package  (MEPCMP)  is  a  C+- f-  package  which  can  generate  high  di¬ 
mensional,  multi-element  collocation  points  based  on  arbitrary  probability  density  function  for  the 
application  of  Multi-Element  Probabilistic  Collocation  Method  (MEPCM). 

•  Patera’s  group  -  Several  technology  and  software  disclosures  were  made  to  the  MIT  Technology 
Licensing  Office  during  the  life  of  the  grant.  The  MIT  Office  of  Sponsored  Research  will  file  the 
official  report  on  intellectual  property.  We  note  here  a  transition  recipient. 

Transition  Organization:  Akselos,  Inc  http :  / / www .  akselos .  com/ company .  html 
Point  of  contact  within  Akselos: 
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Thomas  Leurent,  CEO 
Akselos  S.A. 

EPFL  Innovation  Park,  Building  D 

1015  Lausanne 

Switzerland 

E  t hornas .  leurent  @akselos .  com 

Technology  Licensed  by  Akselos:  MIT  Case  No.  15007  “scRBE”  by  Harriet  Li,  Dinh  Bao 
Phuong  Huynh,  David  John  Knezevic,  and  Anthony  T  Patera. 

MIT  Case  No.  15408  “SCRBE  VI. 1  Software”,  by  David  John  Knezevic. 

•  Hesthaven’s  group  -  Working  closely  with  HyperComp,  Inc  to  transition  the  use  of  reduced  basis 
methods  and  related  UQ  technology  into  TEMPUS  software.  TEMPUS  is  used  widely  by  AFRL. 

PhD  student  X.  Zhu  spent  the  summer  at  HyperComp  as  an  intern  working  on  transitioning  reduced 
order  modeling  ideas  and  software  into  HyperComp  Inc. 
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Abstract 

We  developed  an  integrated  methodology  for  uncertainty  quantification  (UQ)  that  proceeds  from  initial 

problem  definition  to  engineering  applications.  Towards  this  goal,  we  worked  on  five  research  areas:  (1 ) 

Mathematical  analysis  of  Stochastic  Partial  Differential  Equations  (SPDEs)  and  multiscale  formulation;  (2) 

Numerical  solution  of  SPDEs;  (3)  Reduced-Order  modeling;  (4)  Estimation/Inverse  problems;  and  (5) 

Robust  optimization  and  control.  This  work  sets  the  mathematical  foundations  of  Uncertainty  Quantification 

methods  used  by  many  diverse  communities  in  computational  mechanics,  fluid  dynamics,  plasma 

dynamics,  and  materials  science.  We  have  pioneered  methods  for  efficient  high-dimensional 

representations  of  stochastic  processes,  established  Wick-Malliavin  approximation  for  nonlinear  SPDEs, 

theoretical  error  estimates  for  multiscale  parametric  and  stochastic  PDEs,  a  new  approach  to  design  of 

experiment  and  UQ  on  parametric  manifolds,  multi-fidelity  optimization-under-uncertainty,  a  data-driven 

Bayasian  framework  and  probabilistic  graphical  models  for  UQ,  and  information-based  coarse  graining 

methods.  We  have  also  demonstrated  an  integration  of  our  UQ  methodology  and  all  five  areas  for  a 

benchmark  problem.  We  have  published  more  than  1 50  papers  in  top  mathematical  journals,  obtained  one 

patent  (MIT),  and  have  established  one  software  company  (MIT). 
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Specific  research  highlights  include: 


Mathematical  Theory:  Quantization-renormalization  of  SPDEs;  New  evolution  equations  for  joint-pdf  of 
SPDEs;  Nonlinear  Malliavin  calculus. 

Reduced  Basis  Methods  (RBM):  Integral  equations  and  multi-scattering  problems;  Robust  design, 
parameter  estimation,  and  model  uncertainty. 

Adaptive  ANOVA:  Convergence  theory;  Parameter  compression  and  RBM;  Fluid  flows,  porous  media, 
multi-scattering. 

Bayesian  Framework:  coarse-g raining;  Active  learning  +  SPDEs;  Adaptive  SMC,  dependent  random 
variables,  Model  uncertainty  in  inverse  problems. 

Numerical  SPDEs:  Data-driven  stochastic  multiscale  method,  Multiscale  multilevel  MC,  Probabilistic 
graphical  models,  Long-time  integrators  of  SPDEs. 

Software:  MEPCM  library  (polynomial  chaos  +  ANOVA);  RBM  libraries  -  RBOOMIT,  RBAppMIT;  Akselos, 

Inc;  Random  poly-crystals  -  RPCrystal;  TEMPUS/Hypercomp. 

In  addition,  we  have  extended  the  original  proposal  to  introduce  new  concepts  as  follows: 

Stochastic  Delay  ODEs  &  PDEs 

Fractional  PDEs  -  First  Symposium  on  June  3-5,  201 3  (Newport,  Rl) 

Reduced  Basis  for  design  of  experiments 
Multiscale  high-order  FEM  and  Reduced  Basis 
A  control-theoretic  approach  to  “inference  for  prediction” 

UQfor  polycrystals  using  microstructure  images  and  FFT 

Probabilistic  graphical  models  for  UQ 

Bayesian  surrogates  to  compute  epistemic  uncertainty 

Distribution  Statement 

This  is  block  12  on  the  SF298  form. 

Distribution  A  -  Approved  for  Public  Release 

Explanation  for  Distribution  Statement 

If  this  is  not  approved  for  public  release,  please  provide  a  short  explanation.  E.g.,  contains  proprietary  information. 
SF298  Form 

Please  attach  your  SF298  form.  A  blank  SF298  can  be  found  here.  Please  do  not  password  protect  or  secure  the  PDF 
The  maximum  file  size  for  an  SF298  is  50MB. 

AFD-070820-035.pdf 

Upload  the  Report  Document.  File  must  be  a  PDF.  Please  do  not  password  protect  or  secure  the  PDF  .  The 
maximum  file  size  for  the  Report  Document  is  50MB. 

FINAL_FA9550-09-1  -061 3.pdf 

Upload  a  Report  Document,  if  any.  The  maximum  file  size  for  the  Report  Document  is  50MB. 

Archival  Publications  (published)  during  reporting  period: 

Changes  in  research  objectives  (if  any): 

Change  in  AFOSR  Program  Manager,  if  any: 

Old  Program  Manager:  Dr.  Fariba  Fahrool 

DISTRIBUTION  A:  Distribution  approved  for  public  release 


New  Program  Manager:  Dr.  Jean-Luc  Cambier 

Extensions  granted  or  milestones  slipped,  if  any: 

AFOSR  LRIR  Number 

LRIR  Title 

Reporting  Period 

Laboratory  Task  Manager 

Program  Officer 

Research  Objectives 

Technical  Summary 

Funding  Summary  by  Cost  Category  (by  FY,  $K) 


Starting  FY 

FY+1 

FY+2 

Salary 

Equipment/Facilities 

Supplies 

Total 

Report  Document 

Report  Document  -  Text  Analysis 

Report  Document  -  Text  Analysis 

Appendix  Documents 

2.  Thank  You 

E-mail  user 

Nov  24,  201 5  14:21 :1 2  Success:  Email  Sent  to:  George_Karniadakis@brown.edu 


DISTRIBUTION  A:  Distribution  approved  for  public  release 


